feat(webrtc): STUN-dispatch WebRTC-Direct listener (v1) + v1 dialer, /sdp harness opt-in - #1449
Conversation
25f1532 to
7398e14
Compare
tox commands_pre installed no extras, so aiortc was absent in CI and every test under tests/core/transport/webrtc was skipped on GitHub. aiortc 1.15 is pure Python and its native deps (av, pylibsrtp) ship manylinux/win wheels, so no system packages are needed. Refs libp2p#1437
…oles The listener's inbound completion was a stub: after ICE it logged and returned, so no inbound connection ever reached the handler. dial() was broken too - get_remote_fingerprint read attributes aiortc does not have. - listener: own a trio nursery (system task, like TCP), hop asyncio->trio without blocking the loop, run Noise XX as *initiator* (spec: server initiates, dialer responds), then hand the authenticated connection to the handler; bound in-flight unauthenticated inbounds. - transport.dial(): Noise responder; verify authenticated peer ID against /p2p/ after the handshake. - noise: role-ordered prologue (dialer fingerprint, then server); PatternXX.handshake_outbound(remote_peer=None) skips only the ID equality check; DataChannelReadWriter.read(n) honours n (the Noise packet reader asks for the 2-byte prefix alone). - helpers: get_remote_fingerprint reads pc.sctp.transport._ssl peer cert; noise send waits for data channel 0 to open. - tests: transport-level dial->listen->stream echo loopback, wrong /p2p/ rejection, e2e Noise role test, buffered read test, fingerprint test. Refs libp2p#1437
- Frame Noise handshake bytes as uvarint-prefixed webrtc.pb.Message stream frames on channel 0 (spec 'Multiplexing'; what go/js do) instead of raw bytes - required for interop; py<->py was symmetric so tests did not notice. - Listener: guard the handler call - the nursery lives in a trio system task, so an escaping exception aborted the whole trio run. - dial(): bound the Noise phase (responder's first step is a read) with handshake_timeout; close the PC on every failure path, not just the two explicit mismatches; do not pass config.ice_servers (previously no STUN servers were used; the default Google STUN added ~5s to offline dials). - Tests: framing round-trip/chunking/FIN/malformed, handler-exception regression. Refs libp2p#1437
…connection) aiortc has no injection point for an external ICE connection, so a listener that demuxes on one UDP port could not use RTCPeerConnection on top of UdpMux. Add attach_muxed_connection(pc, mux, conn): swaps the gatherer/transport aioice.Connection for the mux-backed one, rebinds the DTLS _recv/_send that RTCIceTransport captured at construction, and keeps the mux tables in sync on ICE state changes. UdpMux fixes found while validating that path with a real peer: - mark _local_candidates_start so aiortc's gather() from setLocalDescription is a no-op (it bound extra sockets and appended their candidates to the answer); - STUN responses/indications carry no USERNAME - route them by address, and learn peer addresses from inbound checks and outbound sends; - unregister(ufrag) also drops the addresses learned for that protocol; - STUN-shaped-but-malformed datagrams go straight to the connection's data path (StunProtocol only catches ValueError, so struct.error escaped). Test: mux-backed server PC vs plain aiortc client PC on loopback - answer advertises only the shared port, ICE/DTLS/SCTP connect, data flows both ways, tables empty after close. webrtc extra now aiortc>=1.15. Refs libp2p#1437
… mux test - _learn_addr: latest connection wins for a reused (ip, port) (pion behaviour) so a redial from the same socket reaches its new connection; cap learned addresses per protocol (unauthenticated STUN with a live ufrag from many source ports must not grow the table without bound). - test_pc_over_mux: bind the mux on 0.0.0.0 and advertise a real host address - a 127.0.0.1-bound socket cannot reply to a LAN-bound peer socket on Windows (WinError 1231), which Linux's weak-host model hid. Refs libp2p#1437
A hung close() after a primary failure would outlive asyncio.wait_for and be killed by pytest-timeout, reported as an xdist 'worker crashed' that hides the real error (seen on Windows CI). Refs libp2p#1437
7398e14 to
f761dd4
Compare
|
@seetadev @acul71 Rebased on latest main. This PR adds the spec-aligned WebRTC-Direct listener: one shared UDP port, STUN |
…ection) pc.close() deadlocks on Windows CPython 3.12/3.13: DTLS queues its close_notify datagram, transport.close() runs with that write in flight and defers connection_lost to the write callback, but _ProactorDatagramTransport._loop_writing early-returns on _conn_lost - the deferred connection_lost is never delivered and aioice's StunProtocol.close() awaits its closed-future forever (confirmed via task/transport dumps on CI: closing=True, write_fut finished, closed.done=False). close_peer_connection(pc) bounds close() and, on timeout, abort()s the lingering aioice transports - _force_close delivers connection_lost unconditionally - then lets close() finish. Used in the PC-over-mux test; production call sites switch in the listener PR. Refs libp2p#1437
f761dd4 to
86d8699
Compare
|
CI note: the single red check (windows 3.13 core) is |
Writing a wrong - or upgraded-away - private/mangled attribute name is a silent no-op: setattr creates a new attribute the library never reads and the failure only surfaces far downstream (the SDP-fingerprint invariant test, an ICE that binds extra sockets). Assert hasattr at the write so it fails at the line that caused it, and so an aiortc/aioice upgrade that renames or drops a slot breaks loudly at construction. Applied to the DTLS cert pin (_RTCPeerConnection__certificates, ca8331f), the attach_muxed_connection injection points, and add_ice_connection's aioice fields. Refs libp2p#1437
Listener (spec path, default): one shared UDP socket via UdpMux. The first inbound STUN BINDING REQUEST is parsed for USERNAME = server_ufrag:client_ufrag; the libp2p+webrtc+v1/ prefix selects the flow (unknown/missing prefix -> rejected, never assumed v1; both halves validated as ice-chars, 4..256). A mux-backed ICE connection is registered for the ufrag, the packet replayed into it, and an aiortc PC attached (attach_muxed_connection). The dialer's offer is inferred from the packet: ufrag == pwd == server credential, c=/candidate at the STUN source, a=setup:active so aiortc takes the DTLS server role, placeholder fingerprint with DTLS peer verification disabled for inbound per spec (Noise authenticates). Then the existing PR1 completion path (Noise initiator -> handler). In-flight cap on unauthenticated inbounds. Dialer (v1): the same libp2p+webrtc+v1/<random> string is set as ufrag and pwd on the aioice Connection (aiortc regenerates ICE creds from it in setLocalDescription; SDP text munging is ignored) and on a synthetic ICE-Lite, setup:passive answer built from the multiaddr - aiortc pins the server's DTLS cert via the certhash fingerprint. The HTTP POST /sdp harness is now opt-in (WebRTCTransportConfig.enable_sdp_http_harness, off by default): when on, the listener also serves it on TCP and our dialer uses it. sdp.py: parse_direct_username, build_inferred_offer, build_synthetic_answer, make_v1_credential; _generate_ice_credential emits ice-chars only (token_urlsafe produced '-'/'_' which aioice rejects). UdpMux passes the full USERNAME to the unknown-STUN handler. Tests: loopback echo over STUN and over the harness, two concurrent dials on one port, unknown-prefix rejection, in-flight cap, sdp helpers. Refs libp2p#1437
- listen(): bind the UDP socket before spawning the trio nursery and map OSError to WebRTCConnectionError, so a bind failure leaves no orphaned system task. - close(): cancel in-flight inbound setup tasks so their peer connections close immediately instead of after handshake_timeout. - ice-char validation uses fullmatch ($ accepted a trailing newline). - tests: bind listeners on 0.0.0.0 (advertised as 127.0.0.1) - on Windows a 127.0.0.1-bound socket cannot answer a LAN-bound peer socket; add bind-failure and cancel-on-close assertions, newline rejection cases. Refs libp2p#1437
aiortc dialers gather LAN host candidates only (aioice skips loopback) and on Windows a LAN-bound UDP socket cannot send to 127.0.0.1 (WinError 1214), so a listener advertising 127.0.0.1 is unreachable from a Windows aiortc dialer. Bind the test listeners on the first non-loopback IPv4 interface (fallback 127.0.0.1) and target the advertised host from the STUN poke tests (which bind 0.0.0.0 for the same reason). Refs libp2p#1437
… site The Windows proactor close hang (see the close_peer_connection commit) would otherwise stall the listener's inbound failure paths, the dialer's failure path, and WebRTCConnection.close() via _close_pc_cb - each a leaked asyncio task on the bridge loop. Refs libp2p#1437
…_attr Dialer v1 ICE-credential fields on the aioice Connection and the listener's inbound _validate_peer_identity replacement now assert the slot exists before writing, matching the pattern introduced in libp2p#1448. Refs libp2p#1437
86d8699 to
818d90c
Compare
What
Third step for #1437 — the spec-aligned listener.
Listener (default path) — one shared UDP socket (
UdpMux):USERNAME = server_ufrag:client_ufrag;libp2p+webrtc+v1/prefix selects the flow. Unknown/missing prefix → rejected (spec: never assume v1). Both halves validated (ice-chars, 4–256).attach_muxed_connection, feat(webrtc): run aiortc RTCPeerConnection over UdpMux (attach_muxed_connection) #1448).c=/candidate at the STUN source,a=setup:active(so aiortc takes the DTLS server role — withactpassit would answer as client), placeholder fingerprint with DTLS peer verification disabled for inbound per spec step 6.2/7 (Noise authenticates).max_in_flight_connections).Dialer (v1) — sets
libp2p+webrtc+v1/<random>as ufrag and pwd on the aioiceConnection(aiortc regenerates ICE creds from it insetLocalDescription; SDP-text munging is ignored) and on a synthetic ICE-Litea=setup:passiveanswer built from the multiaddr; the certhash fingerprint in that answer makes aiortc pin the server's DTLS cert.HTTP
POST /sdpharness → opt-inWebRTCTransportConfig(enable_sdp_http_harness=True)(experimental, py↔py). Off by default; nothing else used it.sdp.py:parse_direct_username,build_inferred_offer,build_synthetic_answer,make_v1_credential; credentials generated with ice-chars only (token_urlsafeproduced-/_, which aioice rejects).UdpMuxunknown-STUN handler now receives the fullUSERNAME.Not in this PR (tracked on #1437)
libp2p+webrtc+v2/, no munging; specs#715 still open) — listener currently drops v2 first contacts with a debug log.Tests
tests/core/transport/webrtc: 204 passed; loopback file 6× no flake; mypy/pyrefly clean.Stacked on #1448 → #1447 → #1446.
Refs #1437